Accelerating Value Iteration with Anchoring

Value Iteration (VI) is foundational to the theory and practice of modern reinforcement learning, and it is known to converge at a $\mathcal{O}(\gamma^k)$-rate, where $\gamma$ is the discount factor. Surprisingly, however, the optimal rate for the VI setup was not known, and finding a general acceleration mechanism has been an open problem. In this paper, we present the first accelerated VI for both the Bellman consistency and optimality operators. Our method, called Anc-VI, is based on an \emph{anchoring} mechanism (distinct from Nesterov's acceleration), and it reduces the Bellman error faster than standard VI. In particular, Anc-VI exhibits a $\mathcal{O}(1/k)$-rate for $\gamma\approx 1$ or even $\gamma=1$, while standard VI has rate $\mathcal{O}(1)$ for $\gamma\ge 1-1/k$, where $k$ is the iteration count. We also provide a complexity lower bound matching the upper bound up to a constant factor of $4$, thereby establishing optimality of the accelerated rate of Anc-VI. Finally, we show that the anchoring mechanism provides the same benefit in the approximate VI and Gauss--Seidel VI setups as well

TakeCARE, a Video to Promote Bystander Behavior on College Campuses: Replication and Extension

Previous research has demonstrated that college students who view TakeCARE, a video bystander program designed to encourage students to take action to prevent sexual and relationship violence (i.e., bystander behavior), display more bystander behavior relative to students who view a control video. The current study aimed to replicate and extend these findings by testing two different methods of administering TakeCARE and examining moderators of TakeCARE’s effects on bystander behavior. Students at four universities (n = 557) were randomly assigned to one of three conditions: (a) view TakeCARE in a monitored computer lab, (b) view TakeCARE at their own convenience after receiving an email link to the video, or (c) view a video about study skills (control group). Participants completed measures of bystander behavior at baseline and at a 1-month follow-up. Participants in both TakeCARE conditions reported more bystander behavior at follow-up assessments, compared with participants in the control condition. The beneficial effect of TakeCARE did not differ significantly across administration methods. However, the effects of TakeCARE on bystander behavior were moderated by students’ perceptions of campus responsiveness to sexual violence, with more potent effects when students perceived their institution as responsive to reports of sexual violence

Censored Sampling of Diffusion Models Using 3 Minutes of Human Feedback

Diffusion models have recently shown remarkable success in high-quality image generation. Sometimes, however, a pre-trained diffusion model exhibits partial misalignment in the sense that the model can generate good images, but it sometimes outputs undesirable images. If so, we simply need to prevent the generation of the bad images, and we call this task censoring. In this work, we present censored generation with a pre-trained diffusion model using a reward model trained on minimal human feedback. We show that censoring can be accomplished with extreme human feedback efficiency and that labels generated with a mere few minutes of human feedback are sufficient. Code available at: https://github.com/tetrzim/diffusion-human-feedback.Comment: Published in NeurIPS 202

Functional Reciprocity of Amyloids and Antimicrobial Peptides: Rethinking the Role of Supramolecular Assembly in Host Defense, Immune Activation, and Inflammation

Pathological self-assembly is a concept that is classically associated with amyloids, such as amyloid-β (Aβ) in Alzheimer's disease and α-synuclein in Parkinson's disease. In prokaryotic organisms, amyloids are assembled extracellularly in a similar fashion to human amyloids. Pathogenicity of amyloids is attributed to their ability to transform into several distinct structural states that reflect their downstream biological consequences. While the oligomeric forms of amyloids are thought to be responsible for their cytotoxicity via membrane permeation, their fibrillar conformations are known to interact with the innate immune system to induce inflammation. Furthermore, both eukaryotic and prokaryotic amyloids can self-assemble into molecular chaperones to bind nucleic acids, enabling amplification of Toll-like receptor (TLR) signaling. Recent work has shown that antimicrobial peptides (AMPs) follow a strikingly similar paradigm. Previously, AMPs were thought of as peptides with the primary function of permeating microbial membranes. Consistent with this, many AMPs are facially amphiphilic and can facilitate membrane remodeling processes such as pore formation and fusion. We show that various AMPs and chemokines can also chaperone and organize immune ligands into amyloid-like ordered supramolecular structures that are geometrically optimized for binding to TLRs, thereby amplifying immune signaling. The ability of amphiphilic AMPs to self-assemble cooperatively into superhelical protofibrils that form structural scaffolds for the ordered presentation of immune ligands like DNA and dsRNA is central to inflammation. It is interesting to explore the notion that the assembly of AMP protofibrils may be analogous to that of amyloid aggregates. Coming full circle, recent work has suggested that Aβ and other amyloids also have AMP-like antimicrobial functions. The emerging perspective is one in which assembly affords a more finely calibrated system of recognition and response: the detection of single immune ligands, immune ligands bound to AMPs, and immune ligands spatially organized to varying degrees by AMPs, result in different immunologic outcomes. In this framework, not all ordered structures generated during multi-stepped AMP (or amyloid) assembly are pathological in origin. Supramolecular structures formed during this process serve as signatures to the innate immune system to orchestrate immune amplification in a proportional, situation-dependent manner

Functional Reciprocity of Amyloids and Antimicrobial Peptides: Rethinking the Role of Supramolecular Assembly in Host Defense, Immune Activation, and Inflammation

Pathological self-assembly is a concept that is classically associated with amyloids, such as amyloid-β (Aβ) in Alzheimer's disease and α-synuclein in Parkinson's disease. In prokaryotic organisms, amyloids are assembled extracellularly in a similar fashion to human amyloids. Pathogenicity of amyloids is attributed to their ability to transform into several distinct structural states that reflect their downstream biological consequences. While the oligomeric forms of amyloids are thought to be responsible for their cytotoxicity via membrane permeation, their fibrillar conformations are known to interact with the innate immune system to induce inflammation. Furthermore, both eukaryotic and prokaryotic amyloids can self-assemble into molecular chaperones to bind nucleic acids, enabling amplification of Toll-like receptor (TLR) signaling. Recent work has shown that antimicrobial peptides (AMPs) follow a strikingly similar paradigm. Previously, AMPs were thought of as peptides with the primary function of permeating microbial membranes. Consistent with this, many AMPs are facially amphiphilic and can facilitate membrane remodeling processes such as pore formation and fusion. We show that various AMPs and chemokines can also chaperone and organize immune ligands into amyloid-like ordered supramolecular structures that are geometrically optimized for binding to TLRs, thereby amplifying immune signaling. The ability of amphiphilic AMPs to self-assemble cooperatively into superhelical protofibrils that form structural scaffolds for the ordered presentation of immune ligands like DNA and dsRNA is central to inflammation. It is interesting to explore the notion that the assembly of AMP protofibrils may be analogous to that of amyloid aggregates. Coming full circle, recent work has suggested that Aβ and other amyloids also have AMP-like antimicrobial functions. The emerging perspective is one in which assembly affords a more finely calibrated system of recognition and response: the detection of single immune ligands, immune ligands bound to AMPs, and immune ligands spatially organized to varying degrees by AMPs, result in different immunologic outcomes. In this framework, not all ordered structures generated during multi-stepped AMP (or amyloid) assembly are pathological in origin. Supramolecular structures formed during this process serve as signatures to the innate immune system to orchestrate immune amplification in a proportional, situation-dependent manner

Genome of Drosophila suzukii, the spotted wing drosophila.

Drosophila suzukii Matsumura (spotted wing drosophila) has recently become a serious pest of a wide variety of fruit crops in the United States as well as in Europe, leading to substantial yearly crop losses. To enable basic and applied research of this important pest, we sequenced the D. suzukii genome to obtain a high-quality reference sequence. Here, we discuss the basic properties of the genome and transcriptome and describe patterns of genome evolution in D. suzukii and its close relatives. Our analyses and genome annotations are presented in a web portal, SpottedWingFlyBase, to facilitate public access

Wnt activation downregulates olfactomedin-1 in Fallopian tubal epithelial cells:a microenvironment predisposed to tubal ectopic pregnancy

Ectopic pregnancy (EP) occurs when the embryo fails to transit to the uterus and attach to the luminal epithelium of the Fallopian tube (FT). Tubal EP is a common gynecological emergency and more than 95% of EP occurs in the ampullary region of the FT. In humans, Wnt activation and downregulation of olfactomedin-1 (Olfm-1) occur in the receptive endometrium and coincided with embryo implantation in vivo. Whether similar molecular changes happen in the FT leading to EP remains unclear. We hypothesized that activation of Wnt signaling downregulates Olfm-1 expression predisposes to EP. We investigated the spatiotemporal expression of Olfm-1 in FT from non-pregnant women and women with EP, and used a novel trophoblastic spheroid (embryo surrogate)-FT epithelial cell co-culture model (JAr and OE-E6/E7 cells) to study the role of Olfm-1 on spheroid attachment. Olfm-1 mRNA expression in the ampullary region of non-pregnant FT was higher (P0.05) in the follicular phase than in the luteal phase. Ampullary tubal Olfm-1 expression was lower in FT from women with EP compared to normal controls at the luteal phase (histological scoring (H-SCORE)1.30.2 vs 2.40.5; P0.05). Treatment of OE-E6/E7 with recombinant Olfm-1 (0.2-5 g/ml) suppressed spheroid attachment to OE-E6/E7 cells, while activation of Wnt-signaling pathway by Wnt3a or LiCl reduced endogenous Olfm-1 expression and increased spheroid attachment. Conversely, suppression of Olfm-1 expression by RNAi increased spheroid attachment to OE-E6/E7 cells. Taken together, Wnt activation suppresses Olfm-1 expression, and this may predispose a favorable microenvironment of the retained embryo in the FT, leading to EP in humans. © 2012 USCAP, Inc All rights reserved.link_to_OA_fulltex

Sequencing of Tuta absoluta genome to develop SNP genotyping assays for species identification

Tuta absoluta is one of the most devastating pests of fresh market and processing tomatoes. Native to South America, its detection was confined to that continent until 2006 when it was identified in Spain. It has now spread to almost every continent, threatening countries whose economies rely heavily on tomatoes. This insect causes damage to all developmental stages of its host plant, leading to crop losses as high as 80–100%. Although T. absoluta has yet to be found in the USA and China, which makes up a large portion of the tomato production in the world, computer models project a high likelihood of invasion. To halt the continued spread of T. absoluta and limit economic loss associated with tomato supply chain, it is necessary to develop accurate and efficient methods to identify T. absoluta and strengthen surveillance programs. Current identification of T. absoluta relies on examination of morphology and assessment of host plant damage, which are difficult to differentiate from that of native tomato pests. To address this need, we sequenced the genomes of T. absoluta and two closely related Gelechiidae, Keiferia lycopersicella and Phthorimaea operculella, and developed a bioinformatic pipeline to design a panel of 21-SNP markers for species identification. The accuracy of the SNP panel was validated in a multiplex format using the iPLEX chemistry of Agena MassARRAY system. Finally, the new T. absoluta genomic resources we generated can be leveraged to study T. absoluta biology and develop species-specific management strategies.info:eu-repo/semantics/acceptedVersio